Summary:

Refactored the functions-python build so it can get packages from api/src and not only functions-python folder.
Introduced include_api_folders in function_config.json to specify the folder from api/src that should be copied to the build. e.g. this was added to batch_datasets/function_config.json:

  "include_api_folders": ["database_gen"],

Refactored python functions so they can be self-contained, i.e. a functions-python function (e.g. batch_datasets) will contain all packages it needs to run and test. batch_datasets can we loaded directly in the IDE (e.g. Pycharm) and be managed in isolation.

To accomplish this, you need to run this:

scripts/function-python-setup.sh --function_name batch_datasets

or

scripts/function-python-setup.sh --all

This will create folder shared and test_shared in the function/src folder. These folders will contain links to the necessary packages.

Running api-tests.sh --folder functions-python will also call function-python-setup.sh for you before running the tests.

This change was done in prevision of #779 that will need packages coming from api/src

Expected behavior:

Build should be the same as before.
tests should run the same as before

Testing tips:

Tested running the build-tests.yml action on GH and made sure the tests run properly.
Also ran some of the python-functions with function-python-run.sh

Please make sure these boxes are checked before submitting your pull request - thanks!

• Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
• Add or update any needed documentation to the repo
• Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
• Linked all relevant issues
• Include screenshot(s) showing how this pull request works and fixes the issue(s)